{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 新冠肺炎疫情分析\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据导入\n",
    "- 使用2019.12.23-2020.2.21的微博热搜（top50）数据\n",
    "    - 原排行榜小时级更新，为了便于分析，选取每天12:00的数据作为当天搜索排行榜数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 原始数据\n",
    "\n",
    "- 微博热搜1223-0210.csv：原始小时级数据\n",
    "   - 一共近8w条 (79278)，每条数据5个字段\n",
    "   - 格式：【时间 排名 热搜内容 最后时间 上榜时间】"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 305,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396390 (79278, 5)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>最后时间</th>\n",
       "      <th>上榜时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020/2/10 12:00</td>\n",
       "      <td>1</td>\n",
       "      <td>全国累计确诊40171例新冠肺炎</td>\n",
       "      <td>2020/2/10 14:52</td>\n",
       "      <td>2020/2/10 8:04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020/2/10 12:00</td>\n",
       "      <td>2</td>\n",
       "      <td>致84人遭隔离男子被立案侦查</td>\n",
       "      <td>2020/2/10 15:04</td>\n",
       "      <td>2020/2/10 11:16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020/2/10 12:00</td>\n",
       "      <td>3</td>\n",
       "      <td>发改委专家 人口流动是SARS时6倍</td>\n",
       "      <td>2020/2/10 14:58</td>\n",
       "      <td>2020/2/10 10:42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020/2/10 12:00</td>\n",
       "      <td>4</td>\n",
       "      <td>四川禽流感</td>\n",
       "      <td>2020/2/10 18:22</td>\n",
       "      <td>2020/2/10 10:12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020/2/10 12:00</td>\n",
       "      <td>5</td>\n",
       "      <td>上班</td>\n",
       "      <td>2020/2/10 17:46</td>\n",
       "      <td>2020/2/10 7:22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                时间 排名                热搜内容             最后时间             上榜时间\n",
       "0  2020/2/10 12:00  1    全国累计确诊40171例新冠肺炎  2020/2/10 14:52   2020/2/10 8:04\n",
       "1  2020/2/10 12:00  2      致84人遭隔离男子被立案侦查  2020/2/10 15:04  2020/2/10 11:16\n",
       "2  2020/2/10 12:00  3  发改委专家 人口流动是SARS时6倍  2020/2/10 14:58  2020/2/10 10:42\n",
       "3  2020/2/10 12:00  4               四川禽流感  2020/2/10 18:22  2020/2/10 10:12\n",
       "4  2020/2/10 12:00  5                  上班  2020/2/10 17:46   2020/2/10 7:22"
      ]
     },
     "execution_count": 305,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "# 导入小时级数据\n",
    "df_hour = pd.read_csv(open(\"微博热搜1223-0210.csv\", encoding='utf8'), sep=',',dtype=str)\n",
    "df1 = df_hour.filter(like='钟南山', axis=0)\n",
    "#df1 = df_hour.filter(regex='12:00$', axis=0)\n",
    "print(df_hour.size, df_hour.shape)\n",
    "df_hour.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 疫情相关数据\n",
    "\n",
    "- 原始数据是小时级，量大，而且很多热搜内容与疫情无关，不便分析，需要提炼疫情相关的内容，单独分析\n",
    "- 微博热搜1.1-2.21.csv：缩减后的天级数据，仅保留疫情相关的搜索词，人工标注完成\n",
    "    - 一共899条，数据量大规模减少，8w→900条\n",
    "    - 格式【时间 排名 热搜内容 标注1 标注2 上榜时间 最后时间】\n",
    "    - 注：\n",
    "        - 标注1：地域信息，取值范围：全国数据、疫情地图、地区、新闻、人物、医院、物质、军队、新闻、知识、专家\n",
    "        - 标注2：主题信息，取值范围："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 306,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6279 (897, 7)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>896</th>\n",
       "      <td>2020-01-09</td>\n",
       "      <td>7</td>\n",
       "      <td>武汉不明原因肺炎病原体为新型冠状病毒</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>知识</td>\n",
       "      <td>2020/1/9\\n17:36</td>\n",
       "      <td>2020/1/9 8:38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2020-01-09</td>\n",
       "      <td>7</td>\n",
       "      <td>武汉不明原因肺炎病原体为新型冠状病毒</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>疫情-知识</td>\n",
       "      <td>2020/1/9\\n17:36</td>\n",
       "      <td>2020/1/9 8:38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>894</th>\n",
       "      <td>2020-01-11</td>\n",
       "      <td>48</td>\n",
       "      <td>专家解读武汉不明原因肺炎</td>\n",
       "      <td>国内</td>\n",
       "      <td>专家|疫情-知识</td>\n",
       "      <td>2020/1/11 14:50</td>\n",
       "      <td>2020/1/11\\n8:18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>893</th>\n",
       "      <td>2020-01-14</td>\n",
       "      <td>8</td>\n",
       "      <td>泰国发现1例新型冠状病毒病例</td>\n",
       "      <td>国外-泰国</td>\n",
       "      <td>疫情-新增</td>\n",
       "      <td>2020/1/14 20:30</td>\n",
       "      <td>2020/1/14\\n11:42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>892</th>\n",
       "      <td>2020-01-15</td>\n",
       "      <td>14</td>\n",
       "      <td>武汉肺炎不排除有限人传人可能</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>疫情-知识</td>\n",
       "      <td>2020/1/15 13:40</td>\n",
       "      <td>2020/1/15\\n7:52</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间  排名                热搜内容        地域        主题             上榜时间  \\\n",
       "896  2020-01-09   7  武汉不明原因肺炎病原体为新型冠状病毒  国内-湖北-武汉        知识  2020/1/9\\n17:36   \n",
       "895  2020-01-09   7  武汉不明原因肺炎病原体为新型冠状病毒  国内-湖北-武汉     疫情-知识  2020/1/9\\n17:36   \n",
       "894  2020-01-11  48        专家解读武汉不明原因肺炎        国内  专家|疫情-知识  2020/1/11 14:50   \n",
       "893  2020-01-14   8      泰国发现1例新型冠状病毒病例     国外-泰国     疫情-新增  2020/1/14 20:30   \n",
       "892  2020-01-15  14      武汉肺炎不排除有限人传人可能  国内-湖北-武汉     疫情-知识  2020/1/15 13:40   \n",
       "\n",
       "                 最后时间  \n",
       "896     2020/1/9 8:38  \n",
       "895     2020/1/9 8:38  \n",
       "894   2020/1/11\\n8:18  \n",
       "893  2020/1/14\\n11:42  \n",
       "892   2020/1/15\\n7:52  "
      ]
     },
     "execution_count": 306,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import datetime\n",
    "#str_p = '2020/2/7 12:00'\n",
    "#dateTime_p = datetime.datetime.strptime(str_p,'%Y/%m/%d %H:%M')\n",
    "\n",
    "# 导入天级数据，1.30以后标注完整（地域+主题）\n",
    "df_day = pd.read_csv(open(\"微博热搜1.1-2.21.csv\", encoding='gbk'), sep=',',dtype=str)\n",
    "# 时间格式矫正，规范，便于字符串排序\n",
    "df_day['时间'] = df_day['时间'].apply(lambda x:str(datetime.datetime.strptime(x,'%Y/%m/%d %H:%M')).split(' ')[0])#.sort_index()\n",
    "# 按照日期升序排列\n",
    "df_day = df_day.sort_values(by='时间') \n",
    "print(df_day.size, df_day.shape)\n",
    "df_day.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 307,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>373</th>\n",
       "      <td>2020-02-06</td>\n",
       "      <td>11</td>\n",
       "      <td>医护夫妻通过声音眼神认出彼此</td>\n",
       "      <td>国内</td>\n",
       "      <td>人物-医护夫妻</td>\n",
       "      <td>2020/2/6 15:36</td>\n",
       "      <td>2020/2/6\\n10:06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>378</th>\n",
       "      <td>2020-02-06</td>\n",
       "      <td>22</td>\n",
       "      <td>四川内江查获50余万个三无口罩</td>\n",
       "      <td>国内-四川</td>\n",
       "      <td>物资-三无口罩</td>\n",
       "      <td>2020/2/6 15:24</td>\n",
       "      <td>2020/2/6\\n10:32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>377</th>\n",
       "      <td>2020-02-06</td>\n",
       "      <td>19</td>\n",
       "      <td>广东发布二号口罩令</td>\n",
       "      <td>国内-广东</td>\n",
       "      <td>政府-口罩令</td>\n",
       "      <td>2020/2/6 17:18</td>\n",
       "      <td>2020/2/6\\n10:50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>376</th>\n",
       "      <td>2020-02-06</td>\n",
       "      <td>16</td>\n",
       "      <td>纸尿裤内衣工厂转做口罩</td>\n",
       "      <td>国内</td>\n",
       "      <td>物资-口罩|生产</td>\n",
       "      <td>2020/2/6 18:18</td>\n",
       "      <td>2020/2/6\\n11:30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>375</th>\n",
       "      <td>2020-02-06</td>\n",
       "      <td>13</td>\n",
       "      <td>北京减免中小微企业房租</td>\n",
       "      <td>国内-北京</td>\n",
       "      <td>政府-小微企业|房租</td>\n",
       "      <td>2020/2/6 13:54</td>\n",
       "      <td>2020/2/6\\n9:48</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间  排名             热搜内容     地域          主题            上榜时间  \\\n",
       "373  2020-02-06  11   医护夫妻通过声音眼神认出彼此     国内     人物-医护夫妻  2020/2/6 15:36   \n",
       "378  2020-02-06  22  四川内江查获50余万个三无口罩  国内-四川     物资-三无口罩  2020/2/6 15:24   \n",
       "377  2020-02-06  19        广东发布二号口罩令  国内-广东      政府-口罩令  2020/2/6 17:18   \n",
       "376  2020-02-06  16      纸尿裤内衣工厂转做口罩     国内    物资-口罩|生产  2020/2/6 18:18   \n",
       "375  2020-02-06  13      北京减免中小微企业房租  国内-北京  政府-小微企业|房租  2020/2/6 13:54   \n",
       "\n",
       "                最后时间  \n",
       "373  2020/2/6\\n10:06  \n",
       "378  2020/2/6\\n10:32  \n",
       "377  2020/2/6\\n10:50  \n",
       "376  2020/2/6\\n11:30  \n",
       "375   2020/2/6\\n9:48  "
      ]
     },
     "execution_count": 307,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_day.sort_values(by='时间', ascending=False) # 逆序排列\n",
    "df_day[df_day['排名'].isin(['1','2','3'])] # 提取每天的前三名\n",
    "df_day[df_day['时间']=='2020-02-06'].head() # 查看某天的热门搜索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 308,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>时间</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-01-09</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-11</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            排名  热搜内容  地域  主题  上榜时间  最后时间\n",
       "时间                                      \n",
       "2020-01-09   2     2   2   2     2     2\n",
       "2020-01-11   1     1   1   1     1     1"
      ]
     },
     "execution_count": 308,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "#df_new.groupby(['时间']).count().sort_values(by='时间')\n",
    "df_day.groupby(['时间'])['排名'].agg([sum])\n",
    "#df_day['时间'].groupby(['时间']).count().reset_index().rename(columns={'0':'数目'})\n",
    "#df_day['排名'].astype(float).groupby(['时间']).size()\n",
    "#df_day['排名'].astype(float).groupby(by=['时间'])\n",
    "df_day.groupby('时间')['排名'].count().reset_index().rename(columns={'0':'数目'})\n",
    "df_day.groupby('时间').count()[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 趋势分析\n",
    "- 疫情相关热词近期有什么变化？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 疫情热度如何变化？\n",
    "- 每天疫情相关热门搜索词有多少？\n",
    "    - 趋势图见末尾，分析如下：\n",
    "    - 1.19前，只有1-2个热搜，此时疫情刚刚爆发，原因不明，只知道是冠状病毒，不确定是否人传人\n",
    "    - 1.20开始，突然上涨到8个热搜，武汉新增136例，卫健委发布病毒性肺炎高发注意事项，同时疫情已经开始蔓延到省外：浙江发现5例、深圳出现8例（1例确诊）、北京确诊2例，上海也开始加强筛查\n",
    "    - 1.21以后，武汉同济协和医院出现15名医护感染，钟南山站肯定新型冠状病毒肺炎人传人，民众开始高度关注，top 50榜单里至少一半是疫情相关\n",
    "    - 1.23，武汉封城，31个热搜\n",
    "    - 1.25，物资缺乏，除夕夜医护吃泡面，承受极大的压力，接近奔溃，大年初一，政府开始调度资源，医护、军队进入疫区\n",
    "    - 1.26，榜单达到高峰：41个热搜，全国疫情迅速发展，民众焦虑，不知所措\n",
    "    - 1.29，热词数目开始下滑，31个，小汤山医院投入使用，火神山马上完工，虽然各地确诊人数仍然不断增加，但大家没那么害怕\n",
    "    - 2.6，小高峰，35个\n",
    "    - 2.20，减少到15个\n",
    "    - 其他分析。。。\n",
    "- 每天疫情相关热门搜索词平均排名高吗？\n",
    "    - 前期，热词少，突发情况排名较高，如武汉出现疫情\n",
    "    - 中期，平均排名15-20，霸占榜单前几名\n",
    "    - 后期，继续上升到8-10\n",
    "- 每天关注度是如何变化？\n",
    "    - >关注度 = 热门搜索排名累加"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'国内': [823, 21522.0], '国外': [74, 1879.0]}\n",
      "{'湖北': [171, 4569.0, {'武汉': [120, 3316.0], '黄冈': [6, 124.0], '松滋': [1, 5.0], '随州': [1, 3.0], '荆州': [2, 29.0], '孝感': [1, 37.0]}], '浙江': [29, 647.0, {'杭州': [4, 79.0], '龙港': [1, 17.0], '宁波': [1, 42.0], '义乌': [1, 31.0]}], '广东': [23, 635.0, {'深圳': [2, 79.0], '广州': [1, 0.0], '汕头': [2, 42.0], '东莞': [1, 12.0], '中山': [1, 5.0]}], '上海': [12, 326.0, {}], '北京': [28, 781.0, {}], '湖南': [8, 197.0, {'长沙': [2, 47.0], '张家界': [1, 40.0]}], '山东': [12, 271.0, {'济南': [2, 30.0], '济宁': [1, 26.0]}], '福建': [3, 87.0, {}], '四川': [9, 202.0, {'成都': [1, 31.0], '汶川': [1, 22.0]}], '黑龙江': [10, 235.0, {'哈尔滨': [1, 48.0], '齐齐哈尔': [1, 25.0]}], '江苏': [15, 273.0, {'南京': [3, 55.0]}], '重庆': [8, 253.0, {}], '河南': [22, 510.0, {'信阳': [1, 49.0], '郑州': [1, 19.0]}], '江西': [3, 17.0, {}], '海南': [2, 28.0, {}], '河北': [7, 138.0, {'承德': [1, 26.0]}], '山西': [2, 35.0, {}], '香港': [3, 116.0, {}], '安徽': [10, 147.0, {}], '广西': [2, 87.0, {'贵州': [1, 48.0]}], '西藏': [9, 237.0, {}], '新疆': [1, 9.0, {}], '天津': [6, 152.0, {}], '吉林': [2, 23.0, {}], '陕西': [5, 132.0, {'西安': [1, 7.0]}], '内蒙古': [3, 40.0, {'呼和浩特': [1, 9.0]}], '贵州': [2, 32.0, {}], '云南': [1, 0.0, {}], '秦皇岛': [1, 17.0, {}], '青海': [2, 46.0, {}]}\n",
      "{'泰国': [2, 84.0, {}], '日本': [18, 509.0, {}], '美国': [3, 60.0, {}], '英国': [2, 50.0, {}], '韩国': [6, 198.0, {}], '新加坡': [2, 69.0, {}], '法国': [2, 26.0, {}], '德国': [2, 19.0, {}], '巴西': [1, 9.0, {}], '澳大利亚': [1, 1.0, {}], '芬兰': [1, 10.0, {}], '意大利': [1, 31.0, {}], '菲律宾': [1, 24.0, {}], '俄罗斯': [5, 136.0, {}], '欧盟': [1, 7.0, {}], '阿富汗': [1, 8.0, {}], '非洲': [1, 44.0, {}], '游轮': [7, 162.0, {}]}\n",
      "{'武汉': [120, 3316.0], '深圳': [2, 79.0], '广州': [1, 0.0], '长沙': [2, 47.0], '成都': [1, 31.0], '汕头': [2, 42.0], '东莞': [1, 12.0], '黄冈': [6, 124.0], '松滋': [1, 5.0], '杭州': [4, 79.0], '西安': [1, 7.0], '哈尔滨': [1, 48.0], '济南': [2, 30.0], '中山': [1, 5.0], '贵州': [1, 48.0], '承德': [1, 26.0], '龙港': [1, 17.0], '齐齐哈尔': [1, 25.0], '随州': [1, 3.0], '汶川': [1, 22.0], '呼和浩特': [1, 9.0], '南京': [3, 55.0], '宁波': [1, 42.0], '张家界': [1, 40.0], '荆州': [2, 29.0], '信阳': [1, 49.0], '孝感': [1, 37.0], '义乌': [1, 31.0], '郑州': [1, 19.0], '济宁': [1, 26.0]}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "day_info_dict = {} # 趋势信息\n",
    "loc_info_dict = {'total':{}, 'overseas':{}, 'prov':{}, 'city':{}} # 地域信息\n",
    "#for i in df.iteritems(): # iterrows()\n",
    "for i in df_day.iterrows():\n",
    "    # 记录每天的搜索词数目\n",
    "    (dt, rank, query, location, topic) = i[1]['时间'], float(i[1]['排名']), i[1]['热搜内容'], i[1]['地域'], i[1]['主题']\n",
    "    if dt not in day_info_dict:\n",
    "        # 记录排名信息，最大，最小，平均\n",
    "        day_info_dict[dt] = {'query_num':1, 'min_rank':50-rank, 'max_rank':50-rank, 'avg_rank':50-rank}\n",
    "    else:\n",
    "        day_info_dict[dt]['query_num'] += 1 # 热词数目\n",
    "        if rank < day_info_dict[dt]['min_rank']:\n",
    "            day_info_dict[dt]['min_rank'] = 50-rank\n",
    "        if rank > day_info_dict[dt]['max_rank']:\n",
    "            day_info_dict[dt]['max_rank'] = 50-rank\n",
    "        day_info_dict[dt]['avg_rank'] = round((day_info_dict[dt]['query_num']+day_info_dict[dt]['avg_rank']*(\n",
    "            day_info_dict[dt]['query_num']-1))/day_info_dict[dt]['query_num'],2)\n",
    "    # 地域信息 国内-湖北-武汉，国外-日本\n",
    "    loc_list = location.split('-')\n",
    "    loc_len = len(loc_list)\n",
    "    if loc_len > 0: # 国内外信息\n",
    "        area = loc_list[0]\n",
    "        if area not in loc_info_dict['total']:\n",
    "            loc_info_dict['total'][area] = [0, 0] # 频次、累计热度（50-名次）\n",
    "        loc_info_dict['total'][area][0] += 1\n",
    "        loc_info_dict['total'][area][1] += 50-rank # 排名越靠前，权重越大\n",
    "    if loc_len > 1: # 省份信息、国家信息\n",
    "        prov = loc_list[1].split('|')[0] # 复合形式：山东|浙江\n",
    "        k = 'prov'\n",
    "        if area == '国外':\n",
    "            k = 'overseas'\n",
    "        if prov not in loc_info_dict[k]:\n",
    "            loc_info_dict[k][prov] = [0, 0, {}] # 频次、累计热度、城市信息\n",
    "        loc_info_dict[k][prov][0] += 1\n",
    "        loc_info_dict[k][prov][1] += 50-rank # 排名越靠前，权重越大\n",
    "    if loc_len > 2: # 城市信息\n",
    "        city =  loc_list[2]\n",
    "        if city not in loc_info_dict[k][prov][2]:\n",
    "            loc_info_dict[k][prov][2][city] = [0,0]\n",
    "        loc_info_dict[k][prov][2][city][0] += 1\n",
    "        loc_info_dict[k][prov][2][city][1] += 50-rank\n",
    "        # 所有城市\n",
    "        if city not in loc_info_dict['city']:\n",
    "            loc_info_dict['city'][city] = [0,0]\n",
    "        loc_info_dict['city'][city][0] += 1\n",
    "        loc_info_dict['city'][city][1] += 50-rank\n",
    "#print(day_info_dict)\n",
    "out_dict = {'趋势':{'日期':[], '热词数目':[], '最大关注度':[], '最小关注度':[], '平均关注度':[]},\n",
    "            '地域':{}}\n",
    "out_dict['趋势']['日期'] = list(day_info_dict.keys())\n",
    "out_dict['趋势']['热词数目'] = [day_info_dict[dt]['query_num'] for dt in out_dict['趋势']['日期']]\n",
    "out_dict['趋势']['最大关注度'] = [day_info_dict[dt]['max_rank'] for dt in out_dict['趋势']['日期']]\n",
    "out_dict['趋势']['最小关注度'] = [day_info_dict[dt]['min_rank'] for dt in out_dict['趋势']['日期']]\n",
    "out_dict['趋势']['平均关注度'] = [day_info_dict[dt]['avg_rank'] for dt in out_dict['趋势']['日期']]\n",
    "#print(json.dumps(out_dict, ensure_ascii=False))\n",
    "tmp_df = pd.DataFrame(out_dict['趋势'])\n",
    "tmp_df[:10]\n",
    "# 世界地图只支持英文，需要先翻译过去\n",
    "country_dict = {'泰国':'Thailand', '日本':'Japan', '美国':'United States', '英国':'United Kingdom', '韩国':'Korea', '新加坡':'Singapore', '法国':'French',\n",
    "                '德国':'German', '巴西':'Brazil', '澳大利亚':'Australian', '芬兰':'Finland', '意大利':'Italian', '俄罗斯':'Russia', \n",
    "                '伊拉克':'Iraq', '伊朗':'Iran', '叙利亚':'Syria', '以色列':'Israel','巴勒斯坦':'Palestine','蒙古':'Mongolia','朝鲜':'Dem. Rep. Korea',\n",
    "                '印度':'Indian', '阿富汗':'Afghanistan', '加拿大':'Canadia', '中国':'China'}\n",
    "out_dict['地域']['总体'] = loc_info_dict['total']\n",
    "out_dict['地域']['世界'] = {\n",
    "    '国家':[country_dict[k] for k,v in loc_info_dict['overseas'].items() if k in country_dict],\n",
    "    '频次':[v[0] for k,v in loc_info_dict['overseas'].items() if k in country_dict],\n",
    "    '关注度':[v[1] for k,v in loc_info_dict['overseas'].items() if k in country_dict],\n",
    "}\n",
    "# 补充中国总体信息\n",
    "out_dict['地域']['世界']['国家'].append('China')\n",
    "out_dict['地域']['世界']['频次'].append(loc_info_dict['total']['国内'][0])\n",
    "out_dict['地域']['世界']['关注度'].append(loc_info_dict['total']['国内'][1])\n",
    "# 省份信息\n",
    "out_dict['地域']['中国'] = {\n",
    "    '省份':[k for k,v in loc_info_dict['prov'].items()],\n",
    "    '频次':[v[0] for k,v in loc_info_dict['prov'].items()],\n",
    "    '关注度':[v[1] for k,v in loc_info_dict['prov'].items()],\n",
    "}\n",
    "# 湖北城市信息\n",
    "out_dict['地域']['湖北'] = {\n",
    "    '城市':[k for k,v in loc_info_dict['prov']['湖北'][2].items()],\n",
    "    '频次':[v[0] for k,v in loc_info_dict['prov']['湖北'][2].items()],\n",
    "    '关注度':[v[1] for k,v in loc_info_dict['prov']['湖北'][2].items()],\n",
    "}\n",
    "# 全国城市信息\n",
    "out_dict['地域']['城市'] = {\n",
    "    '城市':[k for k,v in loc_info_dict['city'].items()],\n",
    "    '频次':[v[0] for k,v in loc_info_dict['city'].items()],\n",
    "    '关注度':[v[1] for k,v in loc_info_dict['city'].items()],\n",
    "}\n",
    "\n",
    "print(loc_info_dict['total'])\n",
    "print(loc_info_dict['prov'])\n",
    "print(loc_info_dict['overseas'])\n",
    "print(loc_info_dict['city'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 主题分析\n",
    "- 热搜内容分布在不同主题里\n",
    "    - 疫情数据：全国确诊人数、疫情地图\n",
    "    - 医护\n",
    "    - 专家\n",
    "    - 政府措施\n",
    "    - 其他\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 以全国确诊人数为例，提取“全国累计确诊”前缀的热搜"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 310,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-310-b8f0e5aa866d>:17: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_c_total['确诊人数'] = df_c_total['热搜内容'].apply(get_number)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "      <th>确诊人数</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>686</th>\n",
       "      <td>2020-01-27</td>\n",
       "      <td>5</td>\n",
       "      <td>全国累计确诊新型肺炎2744例</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-播报</td>\n",
       "      <td>2020/1/27 18:04</td>\n",
       "      <td>2020/1/27\\n8:08</td>\n",
       "      <td>2744</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>649</th>\n",
       "      <td>2020-01-28</td>\n",
       "      <td>1</td>\n",
       "      <td>全国累计确诊病例4515例</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-播报</td>\n",
       "      <td>2020/1/28 19:02</td>\n",
       "      <td>2020/1/28\\n10:40</td>\n",
       "      <td>4515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>2020-01-29</td>\n",
       "      <td>5</td>\n",
       "      <td>全国累计确诊新型肺炎5974例</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-播报</td>\n",
       "      <td>2020/1/29 16:34</td>\n",
       "      <td>2020/1/29\\n8:28</td>\n",
       "      <td>5974</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>588</th>\n",
       "      <td>2020-01-30</td>\n",
       "      <td>4</td>\n",
       "      <td>全国累计确诊新型肺炎7711例</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-播报</td>\n",
       "      <td>2020/1/30 16:18</td>\n",
       "      <td>2020/1/30\\n8:04</td>\n",
       "      <td>7711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>555</th>\n",
       "      <td>2020-01-31</td>\n",
       "      <td>2</td>\n",
       "      <td>全国累计确诊9692例</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>疫情-播报</td>\n",
       "      <td>2020/1/31 13:20</td>\n",
       "      <td>2020/1/31\\n8:28</td>\n",
       "      <td>9692</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间 排名             热搜内容        地域     主题             上榜时间  \\\n",
       "686  2020-01-27  5  全国累计确诊新型肺炎2744例        国内  疫情-播报  2020/1/27 18:04   \n",
       "649  2020-01-28  1    全国累计确诊病例4515例        国内  疫情-播报  2020/1/28 19:02   \n",
       "622  2020-01-29  5  全国累计确诊新型肺炎5974例        国内  疫情-播报  2020/1/29 16:34   \n",
       "588  2020-01-30  4  全国累计确诊新型肺炎7711例        国内  疫情-播报  2020/1/30 16:18   \n",
       "555  2020-01-31  2      全国累计确诊9692例  国内-湖北-武汉  疫情-播报  2020/1/31 13:20   \n",
       "\n",
       "                 最后时间  确诊人数  \n",
       "686   2020/1/27\\n8:08  2744  \n",
       "649  2020/1/28\\n10:40  4515  \n",
       "622   2020/1/29\\n8:28  5974  \n",
       "588   2020/1/30\\n8:04  7711  \n",
       "555   2020/1/31\\n8:28  9692  "
      ]
     },
     "execution_count": 310,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "df_c_total = df_day[df_day['热搜内容'].str.contains('全国累计确诊*')] # 字符串匹配，查找类似搜索\n",
    "# 提取热搜里的数字\n",
    "pattern_num = re.compile(r'[\\D+]*(\\d+).*')\n",
    "\n",
    "def get_number(text):\n",
    "    \"\"\" 提取数字 \"\"\"\n",
    "    res = pattern_num.match(text)\n",
    "    if not res:\n",
    "        return 0\n",
    "    return res.groups()[0]\n",
    "total_num_dict = {}\n",
    "# for i in df_c_total.iterrows():\n",
    "#     # 记录每天的搜索词数目\n",
    "#     (dt, rank, query) = i[1]['时间'].split()[0], float(i[1]['排名']), i[1]['热搜内容']\n",
    "df_c_total['确诊人数'] = df_c_total['热搜内容'].apply(get_number)\n",
    "out_dict['确诊'] = {'日期':list(df_c_total['时间']), '排名':list(df_c_total['排名']), '确诊人数':list(df_c_total['确诊人数'])}\n",
    "#out_dict['确诊']\n",
    "df_c_total.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>833</th>\n",
       "      <td>2020-01-23</td>\n",
       "      <td>21</td>\n",
       "      <td>疫情地图</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-地图</td>\n",
       "      <td>2020/1/24 22:14</td>\n",
       "      <td>2020/1/22\\n23:54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>803</th>\n",
       "      <td>2020-01-24</td>\n",
       "      <td>26</td>\n",
       "      <td>疫情地图</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-地图</td>\n",
       "      <td>2020/1/24 22:14</td>\n",
       "      <td>2020/1/22\\n23:54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>727</th>\n",
       "      <td>2020-01-26</td>\n",
       "      <td>15</td>\n",
       "      <td>最新疫情地图</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-地图</td>\n",
       "      <td>2020/2/10 14:30</td>\n",
       "      <td>2020/1/25\\n17:12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>687</th>\n",
       "      <td>2020-01-27</td>\n",
       "      <td>9</td>\n",
       "      <td>最新疫情地图</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-地图</td>\n",
       "      <td>2020/2/10 14:30</td>\n",
       "      <td>2020/1/25\\n17:12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>653</th>\n",
       "      <td>2020-01-28</td>\n",
       "      <td>9</td>\n",
       "      <td>最新疫情地图</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-地图</td>\n",
       "      <td>2020/2/10 14:30</td>\n",
       "      <td>2020/1/25\\n17:12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间  排名    热搜内容  地域     主题             上榜时间              最后时间\n",
       "833  2020-01-23  21    疫情地图  国内  疫情-地图  2020/1/24 22:14  2020/1/22\\n23:54\n",
       "803  2020-01-24  26    疫情地图  国内  疫情-地图  2020/1/24 22:14  2020/1/22\\n23:54\n",
       "727  2020-01-26  15  最新疫情地图  国内  疫情-地图  2020/2/10 14:30  2020/1/25\\n17:12\n",
       "687  2020-01-27   9  最新疫情地图  国内  疫情-地图  2020/2/10 14:30  2020/1/25\\n17:12\n",
       "653  2020-01-28   9  最新疫情地图  国内  疫情-地图  2020/2/10 14:30  2020/1/25\\n17:12"
      ]
     },
     "execution_count": 311,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_c_map = df_day[df_day['热搜内容'].str.contains('地图')] # 字符串匹配，查找类似搜索\n",
    "#df_c_map\n",
    "out_dict['疫情地图'] = {'日期':list(df_c_map['时间']), '关注度':list(df_c_map['排名'])}\n",
    "df_c_map.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 312,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>870</th>\n",
       "      <td>2020-01-21</td>\n",
       "      <td>1</td>\n",
       "      <td>武汉15名医务人员感染新型冠状病毒</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>医护|疫情-新增</td>\n",
       "      <td>2020/1/21 18:52</td>\n",
       "      <td>2020/1/21\\n7:36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>2020-01-21</td>\n",
       "      <td>24</td>\n",
       "      <td>朱一龙鼓励医生粉丝</td>\n",
       "      <td>国内</td>\n",
       "      <td>医护</td>\n",
       "      <td>2020/1/21 13:36</td>\n",
       "      <td>2020/1/21\\n7:26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>848</th>\n",
       "      <td>2020-01-22</td>\n",
       "      <td>2</td>\n",
       "      <td>新型冠状病毒来源是野生动物</td>\n",
       "      <td>国内</td>\n",
       "      <td>疫情-知识</td>\n",
       "      <td>2020/1/22 19:34</td>\n",
       "      <td>2020/1/22\\n11:26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>857</th>\n",
       "      <td>2020-01-22</td>\n",
       "      <td>13</td>\n",
       "      <td>协和医院骨科医生自愿支援</td>\n",
       "      <td>国内-湖北-武汉</td>\n",
       "      <td>医护</td>\n",
       "      <td>2020/1/22 15:50</td>\n",
       "      <td>2020/1/22\\n8:24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>865</th>\n",
       "      <td>2020-01-22</td>\n",
       "      <td>31</td>\n",
       "      <td>全副武装的白衣战士</td>\n",
       "      <td>国内</td>\n",
       "      <td>医护</td>\n",
       "      <td>2020/1/22 16:08</td>\n",
       "      <td>2020/1/22\\n7:10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间  排名               热搜内容        地域        主题             上榜时间  \\\n",
       "870  2020-01-21   1  武汉15名医务人员感染新型冠状病毒  国内-湖北-武汉  医护|疫情-新增  2020/1/21 18:52   \n",
       "875  2020-01-21  24          朱一龙鼓励医生粉丝        国内        医护  2020/1/21 13:36   \n",
       "848  2020-01-22   2      新型冠状病毒来源是野生动物        国内     疫情-知识  2020/1/22 19:34   \n",
       "857  2020-01-22  13       协和医院骨科医生自愿支援  国内-湖北-武汉        医护  2020/1/22 15:50   \n",
       "865  2020-01-22  31          全副武装的白衣战士        国内        医护  2020/1/22 16:08   \n",
       "\n",
       "                 最后时间  \n",
       "870   2020/1/21\\n7:36  \n",
       "875   2020/1/21\\n7:26  \n",
       "848  2020/1/22\\n11:26  \n",
       "857   2020/1/22\\n8:24  \n",
       "865   2020/1/22\\n7:10  "
      ]
     },
     "execution_count": 312,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_day[df_day['热搜内容'].str.contains('[医生|护士]')].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 疫情内容分析\n",
    "- 采用NLP技术肢解热门搜索词，尝试挖掘内部隐藏的信息\n",
    "    - 参考：[Python的jieba分词及TF-IDF和TextRank 算法提取关键字](https://blog.csdn.net/bozhanggu2239/article/details/80157305)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 313,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word</th>\n",
       "      <th>pos</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>922</th>\n",
       "      <td>肺炎</td>\n",
       "      <td>n</td>\n",
       "      <td>191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>634</th>\n",
       "      <td>新增</td>\n",
       "      <td>v</td>\n",
       "      <td>130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712</th>\n",
       "      <td>武汉</td>\n",
       "      <td>ns</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>确诊</td>\n",
       "      <td>v</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>631</th>\n",
       "      <td>新冠</td>\n",
       "      <td>nz</td>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    word pos  count\n",
       "922   肺炎   n    191\n",
       "634   新增   v    130\n",
       "712   武汉  ns    102\n",
       "861   确诊   v     85\n",
       "631   新冠  nz     68"
      ]
     },
     "execution_count": 313,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#import requests\n",
    "import pandas as pd\n",
    "import datetime\n",
    "import re\n",
    "import jieba\n",
    "from jieba import analyse\n",
    "import jieba.posseg as pseg\n",
    "#import matplotlib.pyplot as plt\n",
    "#import jieba.analyse\n",
    "\n",
    "# 分词\n",
    "# sent_words = [list(jieba.cut(sent0)) for sent0 in sentences]\n",
    "# document = [\" \".join(sent0) for sent0 in sent_words]\n",
    "# print(document)\n",
    "# 自定义词库\n",
    "#jieba.set_dictionary(file_name)\n",
    "#jieba.load_userdict(file_name) # file_name 为文件类对象或自定义词典的路径\n",
    "entity_list = ['新型冠状病毒/8/nz', '冠状病毒/8/nz','疫情地图/8/nz','武汉肺炎/8/nz','人传人','病毒/8/nz','疫情/8/nz',\n",
    "               '病例','新冠/8/nz', 'SARS/8/nz','隔离', '口罩', '卫健委/5/nt','双黄连/8/nz','埃博拉/3/nz','瑞德西韦/5/nz',\n",
    "           '金银潭/5/nz','百步亭/5/nz','火神山/5/nz','雷神山/5/nz','张家界/5/ns', '小汤山/5/nz','威斯特丹号/5/nz','钻石公主号/5/nz',\n",
    "               '方舱医院/5/nz','同济医院/3/nz','协和医院/3/nz','紫外线', '蝙蝠/3/nz','酒精','消毒','确诊','连闯/2/v',\n",
    "               '康复/3/n','封城/3/n','汤圆/3/n', '武软/3/nt', '红十字会/3/nt', '马拉松/3/n', '小姐姐/3/r',\n",
    "               '医护人员/5/nr','医疗队/5/nr','医院院长/5/nr','张定宇/5/nr','护士/5/nr','医生/5/nr',\n",
    "               '钟南山/5/nr', '李兰娟/5/nr','李文亮/5/nr', '应勇/5/nr','王贺胜/5/nr', '王忠林/5/nr']\n",
    "for w in entity_list:\n",
    "    #print(w)\n",
    "    w_list = w.split('/')\n",
    "    freq, tag = 1, None\n",
    "    if len(w_list) > 1:\n",
    "        # [word freq]\n",
    "        freq = w_list[1]\n",
    "    if len(w_list) > 2:\n",
    "        # [word freq tag]\n",
    "        tag = w_list[2]\n",
    "    #jieba.add_word(w)\n",
    "    jieba.add_word(w_list[0], freq=freq, tag=tag)\n",
    "\n",
    "word_list = []\n",
    "for index, row in df_day.iterrows():\n",
    "    content = row[2]\n",
    "    #TextRank 关键词抽取，只获取固定词性\n",
    "    #words = analyse.textrank(content, topK=50, withWeight=False, allowPOS=('ns', 'n', 'vn', 'v'))\n",
    "    #words = analyse.textrank(content, topK=50)\n",
    "    # res = analyse.extract_tags(text) # TF-IDF关键词提取\n",
    "    # 不用关键词工具，因为无法导入自定义词库\n",
    "    for w in pseg.cut(content):\n",
    "        if not ( w.flag.startswith('n') or w.flag in ('v','vn','x')):\n",
    "            continue\n",
    "        if len(w.word) == 1:\n",
    "            continue\n",
    "        # 记录全局分词\n",
    "        word_list.append({'word':w.word,'pos':w.flag, 'count':1})\n",
    "df_kw = pd.DataFrame(word_list)\n",
    "# 词频统计\n",
    "df_kw = df_kw.groupby(['word','pos'])['count'].sum().reset_index().sort_values(by='count', ascending=False)\n",
    "df_kw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 314,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>时间</th>\n",
       "      <th>排名</th>\n",
       "      <th>热搜内容</th>\n",
       "      <th>地域</th>\n",
       "      <th>主题</th>\n",
       "      <th>上榜时间</th>\n",
       "      <th>最后时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>2020-02-14</td>\n",
       "      <td>36</td>\n",
       "      <td>女子连闯3个防疫卡点</td>\n",
       "      <td>国内</td>\n",
       "      <td>群众-违规</td>\n",
       "      <td>2020/2/14 16:06</td>\n",
       "      <td>2020/2/14\\n11:42</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             时间  排名        热搜内容  地域     主题             上榜时间              最后时间\n",
       "159  2020-02-14  36  女子连闯3个防疫卡点  国内  群众-违规  2020/2/14 16:06  2020/2/14\\n11:42"
      ]
     },
     "execution_count": 314,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_day[df_day['热搜内容'].str.contains('连闯')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 315,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "肺炎\t新增\t武汉\t确诊\t新冠\t病例\t湖北\t疫情\t口罩\t诊病\t累计\t全国\t患者\t新型冠状病毒\t浙江\t北京\t医院\t感染\t河南\t隔离\t复工\t疫情地图\t首例\t广东\t回应\t日本\t病毒\t钟南山\t医生\t死亡\t出院\t乘客\t中国\t江苏\t专家\t物资\t上海\t防控\t医护人员\t出现\t山东\t防疫\t安徽\t抗疫\t发现\t火神山\t治愈\t检测\t四川\t企业\t加油\t医疗\t黑龙江\t方舱医院\t驰援\t重庆\t连降\t疑似\t护士\t人员\t隐瞒\t取消\t西藏\t卫健委\t开学\t医疗队\t要求\t医用\t钻石公主号\t可能\t邮轮\t河北\t防护\t硬核\t解放军\t感染者\t医护\t天津\t核酸\t病人\t武汉市\t李兰娟\t重症\t聚集\t韩国\t湖南\t工作\t医院院长\t男子\t双黄连\t金银潭\t救治\t捐赠\t黄冈\t疫苗\t居家\t地区\t时间\t治疗\t不力\n"
     ]
    }
   ],
   "source": [
    "print('\\t'.join(df_kw['word'][:100]))\n",
    "df_day[df_day['热搜内容'].str.contains('连闯')]\n",
    "#df_kw[df_kw['word'].str.contains('小汤山')]\n",
    "# 关键词分门别类\n",
    "tag_dict = {'n':'名词', 'nr':'人物', 'ns':'地名','nz':'专名', 'nt':'机构组织', 'v':'动词', 'vn':'动作'}\n",
    "# 词性：n,v,ns,nr,vn,nt,nz,x,ng,nrt\n",
    "df_kw[df_kw.pos=='nr'][:50] # 人名\n",
    "#df_kw[df_kw.pos=='ns'][:50] # 各地地名\n",
    "#df_kw[df_kw.pos=='n'][:50] # 名词\n",
    "#df_kw[df_kw.pos=='v'][:50] # 动词\n",
    "#df_kw.groupby(['pos']).count().sort_values(by='count', ascending=False)\n",
    "df_kw[df_kw.pos=='vn'][:50] # 动作\n",
    "df_kw[df_kw.pos=='nt'][:50] # 各机构\n",
    "out_dict['关键词'] = {}\n",
    "for tag in tag_dict:\n",
    "    out_dict['关键词'][tag] = {'含义': tag_dict[tag], \n",
    "                '数据':df_kw[['word','count']][df_kw.pos==tag].values.tolist()}\n",
    "#out_dict['关键词']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['李文亮', '钟南山', '传人', '承认', '现象']\n",
      "钟南山/nr\t承认/v\t人/n\t传人/n\t现象/n\t，/x\t李文亮/nr\t说/v\n",
      "钟南山\t医生\t医护人员\t护士\t医疗队\t李兰娟\t医院院长\t韩红\t李文亮\t王辰\t王贺胜\t黄祥龙\t钱七虎\t齐齐哈尔\t齐喊\t黑名单\t洪山\t胡绍任\t陶勇\t解密\t阿富汗\t阿婆\t萨日朗\t蒋超良\t谢娜\t纪检监察\t谢谢\t秦皇岛\t司法厅\t卫星\t周琪\t大吉\t太贵\t唐志红\t任城\t余姚\t刘传健\t丁香\t慈善\t戴法\t承德\t朱一龙\t杜绝\t柳叶刀\t显微镜\t晋升\t富士康\t寻超\t小学生\t少林寺\t岳云鹏\t宝宝\t张流波\t张挪富\t康复者\t张定宇\n"
     ]
    }
   ],
   "source": [
    "import jieba.posseg as pseg\n",
    "#df_kw\n",
    "content = '钟南山承认人传人现象，李文亮说了'\n",
    "words = analyse.textrank(content, topK=50) \n",
    "words = analyse.extract_tags(content, topK=50)\n",
    "print(words)\n",
    "pos = [ '{}/{}'.format(w.word,w.flag) for w in pseg.cut(content) if w.flag.startswith('n') or w.flag in ('v','vn','x')]\n",
    "print('\\t'.join(pos))\n",
    "print('\\t'.join(df_kw[df_kw.pos=='nr']['word']))\n",
    "#df_kw[['word','count']][df_kw.pos=='nr'][:100].values.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['湖北', 4569.0],\n",
       " ['浙江', 647.0],\n",
       " ['广东', 635.0],\n",
       " ['上海', 326.0],\n",
       " ['北京', 781.0],\n",
       " ['湖南', 197.0],\n",
       " ['山东', 271.0],\n",
       " ['福建', 87.0],\n",
       " ['四川', 202.0],\n",
       " ['黑龙江', 235.0],\n",
       " ['江苏', 273.0],\n",
       " ['重庆', 253.0],\n",
       " ['河南', 510.0],\n",
       " ['江西', 17.0],\n",
       " ['海南', 28.0],\n",
       " ['河北', 138.0],\n",
       " ['山西', 35.0],\n",
       " ['香港', 116.0],\n",
       " ['安徽', 147.0],\n",
       " ['广西', 87.0],\n",
       " ['西藏', 237.0],\n",
       " ['新疆', 9.0],\n",
       " ['天津', 152.0],\n",
       " ['吉林', 23.0],\n",
       " ['陕西', 132.0],\n",
       " ['内蒙古', 40.0],\n",
       " ['贵州', 32.0],\n",
       " ['云南', 0.0]]"
      ]
     },
     "execution_count": 317,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_day[df_day['热搜内容'].str.contains('连闯')]\n",
    "df_day.values.tolist()\n",
    "#list(zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国']['关注度']))\n",
    "[ list(a) for a in zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国']['关注度'])][:28]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 322,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\wqw\\\\Desktop\\\\code\\\\mycode\\\\wangqiwen\\\\demo\\\\SARI\\\\BDD\\\\wqw_sari.html'"
      ]
     },
     "execution_count": 322,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#先安装pip install pyecharts==0.5.1\n",
    "#from IPython.display import HTML, SVG\n",
    "# from IPython.core.magic import register_cell_magic\n",
    "import pyecharts as pe\n",
    "\n",
    "page = pe.charts.Page('疫情搜索词分析')\n",
    "opts = pe.options # 配置信息\n",
    "t1 = '疫情搜索词分析（每天12:00的数据）'\n",
    "s1 = '2020-2-24'\n",
    "# 图像数据\n",
    "image = pe.components.Image()\n",
    "img_src = (\n",
    "    \"http://inews.gtimg.com/newsapp_match/0/11297270903/0\"\n",
    "    #\"http://pics1.baidu.com/feed/72f082025aafa40fb2156a3decbe23497af0195b.jpeg\"\n",
    ")\n",
    "image.add(\n",
    "    src=img_src,\n",
    "    style_opts={\"width\": \"60%\", \"height\": \"100%\", \"style\": \"margin-top: 20px\"},\n",
    ").set_global_opts(\n",
    "    title_opts=pe.options.ComponentTitleOpts(title=\"武汉加油\", subtitle=s1)\n",
    ")\n",
    "page.add(image)\n",
    "# 表格数据\n",
    "table = pe.components.Table()\n",
    "table.add(list(df_day.columns), df_day.values.tolist()[:20])\n",
    "table.set_global_opts(title_opts=pe.options.ComponentTitleOpts(title=t1+'(表格)', subtitle=s1))\n",
    "page.add(table)\n",
    "# (1) 每天热搜数目排名变化\n",
    "line_trend = pe.charts.Line()\n",
    "line_trend.set_global_opts(title_opts=opts.TitleOpts(title='疫情热搜趋势分析',pos_left='center',pos_top='8%',subtitle=s1),\n",
    "                          # legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "line_trend.add_xaxis(out_dict['趋势']['日期'])\\\n",
    "    .add_yaxis('疫情热搜-数目', out_dict['趋势']['热词数目'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5),)\\\n",
    "    .add_yaxis('疫情热搜-最大关注度（50-rank）', out_dict['趋势']['最大关注度'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\\\n",
    "    .add_yaxis('疫情热搜-最小关注度（50-rank）', out_dict['趋势']['最小关注度'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\\\n",
    "    .add_yaxis('疫情热搜-平均关注度（50-rank）', out_dict['趋势']['平均关注度'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\n",
    "#page.add(line)\n",
    "bar = pe.charts.Bar()\n",
    "bar.set_global_opts(title_opts=opts.TitleOpts(title='疫情热搜趋势分析',pos_left='center',subtitle=s1),\n",
    "                    legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "bar.add_xaxis(out_dict['趋势']['日期'])\\\n",
    "    .add_yaxis('疫情热搜-数目', out_dict['趋势']['热词数目'],\n",
    "               label_opts=opts.LabelOpts(is_show=False)) # 隐藏标签，避免重影\n",
    "bar.overlap(line_trend)\n",
    "#page.add(bar)\n",
    "page.add(line_trend)\n",
    "# 确诊人数及疫情地图分析\n",
    "line = pe.charts.Line()\n",
    "line.set_global_opts(title_opts=opts.TitleOpts(title='疫情热搜-确诊人数变化',pos_left='center',pos_top='8%',subtitle=s1),\n",
    "                    # legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "line.add_xaxis(out_dict['确诊']['日期'])\\\n",
    "    .add_yaxis('确诊人数', out_dict['确诊']['确诊人数'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\n",
    "page.add(line)\n",
    "# 排名分析--确诊人数\n",
    "line = pe.charts.Line()\n",
    "line.set_global_opts(title_opts=opts.TitleOpts(title='疫情热搜关注度分析',pos_left='center',pos_top='8%',subtitle=s1),\n",
    "                    # legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "line.add_xaxis(out_dict['确诊']['日期'])\\\n",
    "    .add_yaxis('疫情热搜-确诊人数', out_dict['确诊']['排名'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\n",
    "# 排名分析--疫情地图\n",
    "line1 = pe.charts.Line()\n",
    "line1.set_global_opts(title_opts=opts.TitleOpts(title='疫情热搜关注度分析',subtitle=s1),\n",
    "                      #legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "line1.add_xaxis(out_dict['疫情地图']['日期'])\\\n",
    "    .add_yaxis('疫情热搜-疫情地图', out_dict['疫情地图']['关注度'], linestyle_opts=opts.LineStyleOpts(width=5, opacity=0.5))\n",
    "line.overlap(line1)\n",
    "page.add(line)\n",
    "# 地图可视化\n",
    "#   世界地图\n",
    "for k in ('频次', '关注度'):\n",
    "    map_p = pe.charts.Map()\n",
    "    num_range = (min(out_dict['地域']['中国'][k]), sorted(out_dict['地域']['中国'][k])[-2])\n",
    "    t = \"(国内频次：{}, 关注度:{}，国外频次：{}，, 关注度:{})\".format(\n",
    "                                out_dict['地域']['总体']['国内'][0], out_dict['地域']['总体']['国内'][1],\n",
    "                                out_dict['地域']['总体']['国外'][0], out_dict['地域']['总体']['国外'][1])\n",
    "    map_p.set_global_opts(title_opts=opts.TitleOpts(\n",
    "                            title=\"新冠肺炎疫情全球分布-{}\".format(k),\n",
    "                            pos_left='center',subtitle='微博热搜榜单-{}'.format(t)), \n",
    "                          legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle',orient='vertical'), # 靠左居中垂直排列\n",
    "                          toolbox_opts=opts.ToolboxOpts(),\n",
    "                          visualmap_opts=opts.VisualMapOpts(min_=num_range[0], max_=num_range[1]))\n",
    "    map_p.add(k, zip(out_dict['地域']['世界']['国家'], out_dict['地域']['世界'][k]), maptype=\"world\")\n",
    "    map_p.set_series_opts(label_opts=opts.LabelOpts(is_show=False)) # 世界地图不显示label\n",
    "    #map_p.add('关注度', zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国']['关注度']), maptype=\"china\")\n",
    "    page.add(map_p)\n",
    "\n",
    "# 中国地图\n",
    "for k in ('频次', '关注度'):\n",
    "    map_p = pe.charts.Map()\n",
    "    num_range = (min(out_dict['地域']['中国'][k]), sorted(out_dict['地域']['中国'][k])[-2])\n",
    "    t = \"(国内频次：{}, 关注度:{}，国外频次：{}，, 关注度:{})\".format(\n",
    "                                out_dict['地域']['总体']['国内'][0], out_dict['地域']['总体']['国内'][1],\n",
    "                                out_dict['地域']['总体']['国外'][0], out_dict['地域']['总体']['国外'][1])\n",
    "    map_p.set_global_opts(title_opts=opts.TitleOpts(\n",
    "                            title=\"新冠肺炎疫情全国分布-{}\".format(k),\n",
    "                            pos_left='center',subtitle='微博热搜榜单-{}'.format(t)), \n",
    "                          legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle'),\n",
    "                          toolbox_opts=opts.ToolboxOpts(),\n",
    "                          visualmap_opts=opts.VisualMapOpts(min_=num_range[0], max_=num_range[1]))\n",
    "    map_p.add(k, zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国'][k]), maptype=\"china\")\n",
    "    #map_p.add('关注度', zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国']['关注度']), maptype=\"china\")\n",
    "    page.add(map_p)\n",
    "    # 热力图\n",
    "    geo = pe.charts.Geo()\n",
    "    geo.add_schema(maptype=\"china\")\n",
    "    geo.add(k, list(zip(out_dict['地域']['中国']['省份'], out_dict['地域']['中国'][k])),type_=pe.globals.ChartType.HEATMAP)\n",
    "    geo.set_series_opts(label_opts=opts.LabelOpts(is_show=False)).set_global_opts(\n",
    "        visualmap_opts=opts.VisualMapOpts(),\n",
    "        title_opts=opts.TitleOpts(title=\"各省份关注度热力图\"),\n",
    "    )\n",
    "    page.add(geo)\n",
    "# 湖北省内城市地图\n",
    "for k in ('频次', '关注度'):\n",
    "    map_hubei = pe.charts.Map()\n",
    "    num_range = (min(out_dict['地域']['湖北'][k]), max(out_dict['地域']['湖北'][k]))\n",
    "    map_hubei.set_global_opts(title_opts=opts.TitleOpts(\n",
    "                                title=\"新冠肺炎疫情（湖北省内）-{}\".format(k),\n",
    "                                pos_left='center',subtitle='微博热搜榜单'), \n",
    "                              legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle'),\n",
    "                          toolbox_opts=opts.ToolboxOpts(),\n",
    "                          visualmap_opts=opts.VisualMapOpts(min_=num_range[0], max_=num_range[1]))\n",
    "    city_name = ['{}市'.format(i) for i in out_dict['地域']['湖北']['城市']]\n",
    "    map_hubei.add(k, zip(city_name, out_dict['地域']['湖北'][k]), maptype='湖北')\n",
    "    #map_hubei.add('关注度', zip(city_name, out_dict['地域']['湖北']['关注度']), maptype='湖北')\n",
    "    page.add(map_hubei)\n",
    "# 全国城市信息 loc_info_dict['prov']\n",
    "for k in ('频次', '关注度'):\n",
    "    # 区块图（全国城市绘制无效）\n",
    "#     map_hubei = pe.charts.Map()\n",
    "#     num_range = (min(out_dict['地域']['城市'][k]), max(out_dict['地域']['城市'][k]))\n",
    "#     map_hubei.set_global_opts(title_opts=opts.TitleOpts(\n",
    "#                                 title=\"新冠肺炎疫情（全国）-{}\".format(k),\n",
    "#                                 pos_left='center',subtitle='微博热搜榜单'), \n",
    "#                               legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle'),\n",
    "#                           toolbox_opts=opts.ToolboxOpts(),\n",
    "#                           visualmap_opts=opts.VisualMapOpts(min_=num_range[0], max_=num_range[1]))\n",
    "#     city_name = ['{}市'.format(i) for i in out_dict['地域']['城市']['城市']]\n",
    "#     map_hubei.add(k, zip(city_name, out_dict['地域']['城市'][k]), maptype='china')\n",
    "#     #map_hubei.add('关注度', zip(city_name, out_dict['地域']['湖北']['关注度']), maptype='湖北')\n",
    "#     page.add(map_hubei)\n",
    "    # 热力图\n",
    "    geo = pe.charts.Geo()\n",
    "    geo.add_schema(maptype=\"china\")\n",
    "    geo.add(k, list(zip(out_dict['地域']['城市']['城市'], out_dict['地域']['城市'][k])),type_=pe.globals.ChartType.HEATMAP)\n",
    "    geo.set_series_opts(label_opts=opts.LabelOpts(is_show=False)).set_global_opts(\n",
    "        visualmap_opts=opts.VisualMapOpts(),\n",
    "        title_opts=opts.TitleOpts(title=\"全国城市热力图-{}\".format(k)),\n",
    "    )\n",
    "    page.add(geo)\n",
    "\n",
    "# (2) 标题关键词词云\n",
    "#-饼图\n",
    "top_n = 30\n",
    "pie = pe.charts.Pie()\n",
    "pie.set_global_opts(title_opts=opts.TitleOpts(title='疫情热词分析-饼图',pos_left='center',subtitle=s1),\n",
    "                    legend_opts=opts.LegendOpts(is_show=False),\n",
    "                    #legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle'),\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "pie.add(\"饼图可视化（Top {}）\".format(top_n), df_kw[['word','count']].values.tolist()[:top_n])\n",
    "page.add(pie)\n",
    "\n",
    "# 关键词词云可视化\n",
    "wc = pe.charts.WordCloud()\n",
    "wc.set_global_opts(title_opts=opts.TitleOpts(title='疫情热词分析-{}'.format('全部关键词'),\n",
    "                                            pos_left='center',subtitle=s1))\n",
    "wc.add(\"疫情热词分析-所有关键词\", df_kw[['word','count']].values.tolist(), shape='circle') # word_size_range=[10, 500]\n",
    "page.add(wc)\n",
    "# 显示各类词云\n",
    "for tag in tag_dict:\n",
    "    wc = pe.charts.WordCloud()\n",
    "    wc.set_global_opts(title_opts=opts.TitleOpts(title='疫情热词分析-{}'.format(out_dict['关键词'][tag]['含义']),\n",
    "                                                 pos_left='center',subtitle=s1),\n",
    "                       legend_opts=opts.LegendOpts(pos_left='left',pos_top='middle'),\n",
    "                      toolbox_opts=opts.ToolboxOpts())\n",
    "    wc.add(out_dict['关键词'][tag]['含义'], \n",
    "           out_dict['关键词'][tag]['数据'], shape='circle') # word_size_range=[10, 500]\n",
    "    page.add(wc)\n",
    "\n",
    "# 统一绘制、落地\n",
    "out_file = 'wqw_sari.html'\n",
    "page.render(out_file)\n",
    "#page.render_notebook()\n",
    "#HTML(out_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'国家': ['Thailand', 'Japan', 'United States', 'United Kingdom', 'Korea', 'Singapore', 'French', 'German', 'Brazil', 'Australian', 'Finland', 'Italian', 'Russia', 'Afghanistan', 'China'], '频次': [2, 18, 3, 2, 6, 2, 2, 2, 1, 1, 1, 1, 5, 1, 823], '关注度': [84.0, 509.0, 60.0, 50.0, 198.0, 69.0, 26.0, 19.0, 9.0, 1.0, 10.0, 31.0, 136.0, 8.0, 21522.0]}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<pyecharts.charts.basic_charts.map.Map at 0x2cf62840a90>"
      ]
     },
     "execution_count": 319,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "print(out_dict['地域']['世界'])\n",
    "value = [95.1, 23.2, 43.3, 66.4, 88.5]\n",
    "attr = [\"China\", \"Canada\", \"Brazil\", \"Russia\", \"United States\"]\n",
    "map0 = pe.charts.Map()\n",
    "map0.add(\"世界地图\", zip(attr, value), maptype=\"world\")\n",
    "#map0.render(path=\"世界地图.html\")\n",
    "#map0.render_notebook()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['武汉市', '黄冈市', '松滋市', '随州市', '荆州市', '孝感市']\n",
      "[120, 6, 1, 1, 2, 1]\n",
      "[3316.0, 124.0, 5.0, 3.0, 29.0, 37.0]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<pyecharts.charts.basic_charts.map.Map at 0x2cf53679250>"
      ]
     },
     "execution_count": 320,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map_hubei = pe.charts.Map()\n",
    "num_range = (min(out_dict['地域']['湖北']['频次']), max(out_dict['地域']['湖北']['频次']))\n",
    "map_hubei.set_global_opts(title_opts=opts.TitleOpts(\n",
    "                        title=\"新冠肺炎疫情（湖北省内）\",\n",
    "                        subtitle='微博热搜榜单'), \n",
    "                      toolbox_opts=opts.ToolboxOpts(),\n",
    "                      visualmap_opts=opts.VisualMapOpts(min_=num_range[0], max_=num_range[1]))\n",
    "city_name = ['{}市'.format(i) for i in out_dict['地域']['湖北']['城市']]\n",
    "print(city_name)\n",
    "print(out_dict['地域']['湖北']['频次'])\n",
    "print(out_dict['地域']['湖北']['关注度'])\n",
    "map_hubei.add('频次', zip(city_name, out_dict['地域']['湖北']['频次']), maptype='湖北')\n",
    "map_hubei.add('关注度', zip(city_name, out_dict['地域']['湖北']['关注度']), maptype='湖北')\n",
    "#map_hubei.render_notebook()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
